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Background / Context: 

More and more children and youth are bringing to school well-developed patterns of 
adjustment problems in behavior and academics. Antisocial behaviors emerge as early as school 
entry in kindergarten (Hamre & Pianta, 2001; Walker, Colvin, & Ramsey, 1995), tend to be 
stable (Richman, Stevenson, & Graham, 1982), and are likely to increase in severity over time 
(Walker, 1995). Children who enter school with social behavior problems such as oppositional 
behavior and aggression are at elevated risk for continued social difficulties throughout 
elementary school (Campbell & Ewing, 1990; Mesman, Bongers, & Koot, 2001) and for 
exhibiting antisocial behavior during adolescence (Loeber, 1990; Moffitt, 1993; Patterson, Reid, 
& Dishion, 1992). There are well-documented connections between problem behavior during the 
school years and (a) poor academic achievement (McIntosh, Chard, Boland, & Horner, 2006), 

(b) negative academic impact on peers (Sprague & Walker, 2005; Battin-Pearson, et al., 2000) 

(c) negative outcomes into adulthood (Catalano & Hawkins, 1996; Hawkins, et al., 2000), and 

(d) aversive school climate and culture (Gottfredson & Gottfredson, 1985). 

There is empirical evidence demonstrating that school settings that manage behavior and 
maintain a well-developed social climate have a great impact on children’s social and academic 
development (Biglan et al., 2003; Gottfredson et al., 2000; Hawkins et al., 2000). Studies 
demonstrate that behavior programs have a positive effect on variables linked to academic 
performance, e.g. student attendance (Luiselli, Putnam & Sunderland, 2002), time in school due 
to reduced exclusionary disciplinary practices (Putnam, Handler, & O’ Leary -Zonarich, 2003; 
Scott & Barrett, 2004), classroom instructional time (Putnam, Handler, Rey & O’Leary- 
Zonarich, 2002), and academic engagement (Putnam, et al., 2003). Improved behavior support 
has been linked to improved academic outcomes (Larsen, Steele, & Sailor, 2006; 

Luiselli, Putnam, Handler, & Feinberg, 2005; Putnam, et al., 2003). Schools that implement 
school-wide behavior support show greater academic improvements compared to schools where 
school- wide behavior support were not implemented (Homer, Sugai, Todd, & Lewis-Palmer, 
2005, Larsen, et al., 2006, http://www.4j.lane.edu/ess/ebs/data.html). 

The earliest school years (grades K through 3) provide an optimal environment in which 
children can acquire valuable social skills, develop social competence, and practice other 
positive school-related behaviors that are foundational to successful academic experiences. Early 
elementary teachers are ideally positioned to help their students develop these essential skills. 

Yet despite the substantial evidence that effective behavior management practices promote 
desirable social and academic behavior, schools often fail to provide adequate training for 
teachers and staff in this arena (Gottfredson & Gottfredson, 2001; Sprague & Golly, 2004; 
Sprague & Horner, 2006; Sprague & Walker, 2005; Walker, Colvin, & Ramsey, 1995). 

Professional development using one-day inservice training is the standard mechanism by 
which school districts provide on-going training for their teachers, however several limitations 
exist. The standard model of one-day training by an expert does little more than promote 
awareness or motivation, and does little to affect teacher behavior skills development (Bransford, 
et al., 2000; Colvin, Kame’enui, et al., 1993; Hall & Hord, 2001). Secondly, several researchers 
have reported that change is not an event (i.e., in-service activity) but is a process of ongoing 
support, modeling, monitoring, and feedback (Cuban, 1990; Guskey, 1986; Hall & Hord, 2001; 
Hall, George, & Rutherford, 1986). Thirdly, in-service activities are often designed to provide a 
short-term focus, although long-tenn implementation is needed to elicit successful, durable, and 
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substantial behavior or system change (Cuban, 1990; Hall & Hord, 2001; Smylie, 1988; Sugai & 
Homer, 1999). Finally, in-service training is not always available to all staff due to a shortage of 
qualified trainers and lack of funds (Guskey, 1986, Joyce and Showers, 1995). 

Long-standing reports and studies have pointed to the inadequacies of teacher training 
college programs (pre-service), specifying the following concerns: (a) the gap between what is 
taught and what teachers experience, (b) the lack of current contact and experience of college 
instructors with public school needs, (c) the gap between research and practice, (d) the lack of 
practical hands-on strategies, and (e) the inconsistent emphasis on training in classroom 
management from no instruction to high quality instruction (Archer, 1999; Blair, 1999; 
Bransford, Brown, & Cocking, 2000; CEO Forum on Education and Technology, 1999; Landau, 
2001). One of the most disturbing aspects reflecting the inadequacies of pre-service training is 
the on-going problem of retaining teachers (Gonzalez, 1995; McCreight, 2000; Miller, Brownell, 
& Smith, 1999). Thus, there is an urgent need for effective and relevant training in several areas, 
especially in classroom management, at pre-service levels in teacher preparation institutions. 

There is compelling support for early intervention efforts to prevent social adjustment 
problems (Zigler, Taussig, & Black, 1992). The earlier intervention begins, the more likely 
children are to receive long-term benefit (Ramey & Ramey, 1992). Ramey, Bryant, and Suarez 
(1985) conclude that young children are quite responsive to alterations in the quality of their 
environment and to educational efforts. These findings are supported by the longitudinal study 
conducted by Hawkins and colleagues (1999), which showed impressive reductions in antisocial 
behavior through early intervention. 

McIntosh and colleagues (2006) recommend developing behavioral interventions within the 
context of an Rtl model so that students are provided a continuum of instructional and behavioral 
support based on screening measures that indicate response to universal, selected, and intensive 
support. Rtl is an iterative process that involves the use of tiered interventions and assessment to 
document changes in behavior and/or academic outcomes. Rtl represents a thoughtful, child- 
centered approach for offering structured, early interventions through instruction and 
assessments of all students, including those at risk for school failure (NASDSE, 2007). Since the 
recent reauthorization of IDEA (Individuals with Disabilities Education Act, amended in 2004), 
Rtl has become a major stimulus for discussion and action. Rtl has primarily focused on 
academic problems and has been advocated as an alternative approach to special education 
disability determination and early intervention assistance for students who are academically 
unresponsive (Fuchs & Deshler, 2007). 

Focusing on behavior with Rtl for behavior, however, is also important (Sprague, 2006; 
Mcintosh et ah, 2006). The Rtl approach to behavior uses the identical three-tiered logic that is 
used for academics, and this ultimately simplifies the work of schools in both realms — academic 
and behavioral. If students are having a problem with learning, they are more likely than not (and 
sooner or later) going to present problems in behavior, and vice versa. So the effort to evaluate 
and intervene early on both fronts becomes mutually serving for students, families, and 
educators. The mirrored three-tiered structures allow schools to evaluate and intervene for both 
behavioral interventions and academic interventions in an integrated and efficient fashion. 

Rtl is successful when an infrastructure exists to support sufficient assessment and interven- 
tion resources to make decisions that result in successful outcomes for students. School staff 
must possess skills in the necessary assessment and intervention practices. Applying these skills 
requires that staff members have an understanding of evidence-based interventions and how to 
apply them to academic or behavior problems. Additionally, monitoring is needed to assure that 
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interventions are implemented with a high degree of fidelity. Teachers and support services 
personnel will also require the support of building and district administrators and staff to 
implement the Rtl model. Support provided to teachers must extend through the implementation 
of interventions and the collection of appropriate data to assess student progress. Regarding 
social competence as a multi-tiered approach allows teachers to better integrate general and 
special education services, improve the classroom behavioral and instructional climate and be 
better able to identify and assist students who need greater supports (Sprague, Cook, Browning- 
Wright, & Sadler, 2008). 

Purpose / Objective / Research Question / Focus of Study: 

This session will describe the social validity, usability and intent to use of an interactive, 
state-of-the-art, professional development program based on the Response to Intervention (Rtl) 
approach and its core components (e.g., problem-solving strategy; three tiers of intervention 
service delivery with universal, selected and intensive interventions as needed; and integrated 
data collection assessment system to inform instructional decisions at each tier of service 
delivery). We have completed initial development of the online program and have tested its use 
with a sample of K-3 teachers. 

The Student Social Competence program is a behavioral intervention delivered within the 
Rtl framework that provides teachers with effective instructional approaches for promoting 
children’s social competence with the goal of improving the classroom learning environment and 
increasing instructional time. The program provides strategies and interactive tools for 
intervening at universal as well as at selective group levels, and for identifying students with 
disabilities or students needing more intensive, individualized supports. The program is 
supported by a Goal 2 Development grant from the U.S. Department of Education Institute of 
Education Sciences. 

The Student Social Competence program acknowledges the parallelism between 
academics and social outcomes. Academic readiness skills are to academic performance as social 
skills are to social competence. In this context, social competence is a summative, evaluative 
judgment by key social agents (peers, teachers, parents) that one’s behavior is socially effective. 
Social skills are the behavioral indicators that these social agents use in making their overall 
judgments regarding one's competence. In one sense, sociometric ratings and outcomes provide 
such judgments just as achievement tests and progress monitoring do for academic performance. 
The Student Social Competence program provides professional development training for 
teachers and curricular media materials for students. The program includes resources for school 
administrators and behavior teams/coaches that will allow them to implement, support, and 
monitor the use of this program, and to link the training to school district goals through the use 
of the Progress Monitoring tool (PMT). The PMT is an online measurement application that will 
allow teachers to: 1) conduct systematic universal screening for behavioral adjustment, 2) view 
the data in multiple formats (e.g. line graphs, histograms), 3) access tutorials on site usage, 4) 
access resources for working with students who need additional support, 5) notate their work 
with specific students, 6) generate reports and graphical displays of individual student response 
to the parent intervention, and 7) access li nk s to evidence-based resources and interventions. 

Setting: 

Evaluation of the Student Social Competence program and the PMT has involved an 
iterative process which integrates development and research at every stage. The programs have 
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been evaluated by in-house usability testers, in-school usability testers, and by experts. In 2010, 
we perfonned a combined evaluation of the Student Social Competence program and PMT in 40 
classrooms, with K-3 teachers. 

Population / Participants / Subjects: 

Forty K-3 teachers took part in the study. Teachers were recruited through partners at three local 
school districts in Oregon. 

Intervention / Program / Practice: 

Teachers were invited to a 2 hour in-service which explained the purpose and objectives 
of the study, the theoretical background of the student program and PMT, and provided 
instructions for the use the Student Social Competence program and PMT. Participating teachers 
were provided with instructions to obtain credentials to log onto the IRISed.com website as well 
as for the PMT. They were also provided with DVD copies of the student videos, master copies 
of the student activities and other resources. At the in-service, we collected consent forms, 
demographic infonnation, and pre-test data. 

The following week teachers were asked to show at least 3 of the student videos 
(Introduction and 2 skills videos). At the end of the week, the teachers used the PMT to do a 
universal screening of student behavior. Using the data provided by the PMT, teachers identified 
2-3 students who needed additional supports with social skills. The next week teachers used the 
student videos and supplementary materials to provide the identified students with additional 
training and support. The teachers progress monitored these students once a day for a week 
using the PMT. At the conclusion of the week of monitoring, teachers completed the post-test 
questionnaire. 

Research Design: 

The evaluation employed a single-condition, pretest-posttest design. The study’s single- 
condition design does not control for potential threats to internal validity (i.e., extraneous factors; 
Shadish, Cook, & Campbell, 2002). However, it is appropriate for development work. The 
single-condition design allows for the evaluation of feasibility without the added overhead 
required to conduct a randomized trial. The pre-post single-group study allows for evaluation of 
each component and offers the most cost-effective method for evaluating program feasibility. 

Data Collection and Analysis: 

Consent, pre-test data, and demographics of participants were collected via hard copy at the 
in-service. The post-test questionnaires were taken online on Qualtrics. 

Pretest questionnaires contained questions about self-efficacy, concern, knowledge, and 
attitudes towards internet technology. The post-test questionnaire asked the participants about 
self-efficacy, knowledge, satisfaction with the program, and usability. 

Findings / Results: 

The analyses will be conducted to demonstrate feasibility and proof of concept, as well as 
social validity. Paired t-tests will be used to evaluate pre-post gains on the knowledge, attitudes, 
intentions, and self-efficacy. In an effort to examine potential differential effects amongst 
teachers, pre-post gain scores will be correlated with teacher demographic characteristics (e.g., 
teaching experience). Correlational, dose-response analyses will address some of the 
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shortcomings of the single-group, quasi-experimental design and bolster the confidence in the 
internal validity of the study. We will also test the hypothesis that teachers who report greater 
satisfaction with the program will have greater improvement in posttest gain scores. 

Conclusions: 

While many programs have been shown to be effective in research studies, much less is 
known about what it takes to get them implemented well in typical schools (Fixsen, Naoom, 
Blase, Friedman, & Wallace, 2005). There are several important questions to ask while selecting 
and designing behavioral interventions: 

• Is there evidence of effectiveness? When looking at the research, it’s important to ask if 
typical educators are capable of implementing the intervention effectively, or if the researchers 
got the effect only when they ran it. 

• How much does the intervention cost? Interventions that are excessively costly are not 
likely to be used, or even tried. There also may be expensive, ongoing requirements to work with 
the developers of the program. Educators and administrators will want to carefully investigate 
the possibility of any hidden cost. 

• How big an effect should I expect? Many evidence-based practices are considered 
effective even if the improvements are small. As such, it’s important to ask if the practice is 
shown to have a large or small effect. Often with behavioral supports, changes are slow and 
small; unless they understand this and keep focused on the data, staff members can become 
frustrated with the results or the extra efforts required to carry out the intervention. 

• Is there evidence of effectiveness (did typical people guide the intervention) vs., efficacy 
(did the researchers get the effect only when they ran it?). It is typical to conduct research studies 
guided by the developers (and often implemented by them) in order to demonstrate that an 
intervention can work. Once there is evidence of effectiveness, it is important to test the 
intervention in “real world” application in schools. The general finding is that adherence to the 
program will be lower in typical settings, and that the results obtained in the effectiveness trial 
will not be as dramatic. You should carefully consider this when selecting an evidence-based 
intervention. 

• Can teachers integrate the intervention into their daily routine? This may be the most 
important question of all. Many educators are reluctant to adopt behavior support practices that 
do not fit with the daily life of our classrooms. 

The primary aim of our formative research is to address these questions. 
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